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ABSTRACT 

One  of  the  main  challenges  of  co-simulating  hardware-in- 
the-loop  systems  in  real-time  over  the  Internet  is  the  fidelity  of 
the  simulation.  The  dynamics  of  the  Internet  may  significantly 
distort  the  dynamics  of  the  network-integrated  system.  This 
paper  presents  the  development  of  an  iterative  learning  control 
based  approach  to  improve  fidelity  of  such  networked  system 
integration.  Towards  this  end,  a  new  metric  for  characterizing 
fidelity  is  proposed  first,  which,  unlike  some  existing  metrics, 
does  not  require  knowledge  about  the  reference  dynamics  (i.e., 
dynamics  that  would  be  observed,  if  the  system  was  physically 
connected).  Next,  using  this  metric,  the  problem  of  improving 
fidelity  is  formulated  as  an  iterative  learning  control  problem. 
Finally,  the  proposed  approach  is  illustrated  on  a  purely 
simulation-based  case  study.  The  conclusion  is  that  the 
proposed  approach  holds  significant  potential  for  achieving 
high  fidelity  levels. 

Keywords:  real-time  hardware-in-the-loop  simulation, 
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I.  INTRODUCTION 

The  key  benefit  of  hardware-in-the-loop  (HIL)  simulation 
is  well  known:  it  uniquely  combines  the  advantages  of  physical 
prototyping  and  simulation-based  engineering  and  thus  allows 
for  experiments  that  are  at  the  same  time  cost  effective  and 
highly  accurate  [1].  It  has  therefore  become  indispensable  in 


many  application  areas,  such  as  automotive  [2,  3],  aerospace  [4, 
5],  manufacturing  [6],  robotics  [7,  8],  and  defense  [9,  10]. 

Recently,  Internet-distributed  HIL  simulation  (ID-HIL) 
started  attracting  interest  as  a  framework  that  enables 
concurrent  system  engineering  even  if  the  components  that 
comprise  the  desired  HIL  setup  are  geographically  distributed. 
This  idea  has  found  applications  in  the  fields  of  earthquake 
engineering  [11-16]  and  automotive  engineering  [17-23],  and  is 
also  closely  related  to  the  teleoperation  idea  in  robotics  and 
haptics  [24-31]. 

One  of  the  key  challenges  in  ID-HIL  simulation  is  ensuring 
fidelity.  In  this  context,  fidelity  refers  specifically  to  how  close 
the  dynamics  of  the  networked  system  are  to  the  dynamics  that 
would  be  observed  if  the  system  was  physically  integrated  (i.e., 
reference  dynamics).  To  characterize  fidelity,  the  literature  has 
proposed  several  methods.  For  example,  a  frequency-domain 
metric  called  distortion  was  proposed  as  the  normalized 
difference  between  the  networked  and  reference  dynamics  [32, 
33].  In  addition,  the  telerobotics  and  haptics  literatures  have 
proposed  many  other  frequency  domain  metrics,  which  could 
be  adopted  into  the  ID-HIL  framework,  as  well  [34-39].  A  time- 
domain,  statistical  approach  was  also  proposed  to  distinguish 
the  inherent  variation  in  the  reference  dynamics  from  the 
additional  variation  introduced  due  to  the  network  [20,  23]. 

To  improve  fidelity,  the  literature  proposes  several 
methods,  as  well.  These  methods  range  from  selecting 
appropriate  coupling  points  [32,  33]  to  using  feedback  control 
[40-42]  and  observer-based  approaches  [17-19].  The  limitations 
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of  these  techniques  can  be  briefly  summarized  as  follows. 
Coupling-point-based  approaches  work  well,  if  a  coupling  point 
with  desired  characteristics  exists.  However,  such  a  coupling 
point  may  not  always  exist.  Feedback-based  techniques  are 
subject  to  well-known  fundamental  trade-offs  [41-43];  i.e., 
improving  fidelity  at  one  frequency  compromises  it  at  another 
frequency.  Observer-based  techniques  rely  on  the  existence  of 
high-fidelity  models.  However,  not  having  such  models  or 
avoiding  the  need  to  develop  such  models  is  the  main 
motivation  behind  the  HIL  paradigm.  Thus,  improving  fidelity 
in  ID-HIL  systems  is  still  an  open  research  question  and  is  the 
focus  of  this  paper. 

In  an  effort  to  overcome  the  abovementioned  limitations  of 
the  existing  techniques,  this  paper  proposes  an  iterative  learning 
based  approach  to  improving  fidelity  in  ID-HIL  systems.  First, 
a  new  metric  to  characterize  fidelity  is  proposed.  Unlike  some 
of  the  existing  metrics,  e.g.  [20,  23,  32,  33],  this  metric  does  not 
require  knowledge  about  the  reference  dynamics  to  quantify 
fidelity.  Next,  the  paper  formulates  the  fidelity-improvement 
problem  as  an  iterative  learning  control  problem.  Finally,  the 
proposed  formulation  is  applied  to  a  simulation-based  case 
study  that  highlights  the  potential  performance  of  the  proposed 
approach. 

II.  CHARACTERIZING  FIDELITY 

As  mentioned  in  Section  I,  many  metrics  have  been 
proposed  to  characterize  fidelity,  especially  in  telerobotics  and 
haptic  systems.  There  are  two  fundamental  barriers  to  adopting 
the  existing  metrics  to  the  ID-HIL  framework:  (I)  Most  of  these 
metrics  have  been  developed  in  a  linear  and  deterministic 
framework.  However,  ID-HIL  systems  are,  in  general, 
nonlinear  and  stochastic.  (2)  Most  of  the  existing  metrics 
assume  that  the  desired  system  dynamics  are  known  and  utilize 
this  knowledge  as  a  benchmark  to  measure  the  fidelity  of  the 
networked  system.  The  benchmark  for  an  ID-HIL  system  would 
be  a  physical  assembly  of  the  system.  However,  the  main 
motivation  of  the  ID-HIL  paradigm  is  the  lack  of  availability  of 
a  physical  assembly.  Thus,  even  though  the  existing  metrics 
could  still  be  helpful  in  a  research  environment  where  a 
physical  assembly  could  be  made  available  to  develop  and  test 
ID-HIL  techniques,  in  practice  the  desired  system  dynamics  are 
unknown  and  most  of  the  existing  fidelity  metrics  cannot  be 
defined. 

Therefore,  the  aim  of  this  section  is  to  characterize  fidelity 
in  an  ID-HIL  system  despite  the  lack  of  knowledge  about  the 
dynamics  that  the  ID-HIL  system  is  to  emulate  as  accurately  as 
possible.  Towards  this  end,  this  work  uses  the  error  between  the 
instantaneous  values  of  coupling  signals  at  both  ends  of  the 
network. 

To  explain  this  idea  on  an  example,  consider  the  ID-HIL 
framework  shown  in  Fig.  I.  This  is  a  two-site  ID-HIL  example, 
in  which  Systems  1  and  2,  both  of  which  may  include  hardware 
and  models,  are  integrated  over  a  network  through  the  coupling 
variables  q  and  ■  Due  to  the  numerous  considerations 


FIG.  1 .  ILLUSTRATION  OF  DEFINING  THE  ERRORS  IN  THE  COUPLING 
SIGNALS  TOWARDS  CHARACTERIZING  FIDELITY  IN  ID-HIL  ON  AN 
EXAMPLE  WITH  TWO  SITES  (SYSTEM  1  AND  2).  c,  .  REPRESENTS  THE 
/-TH  COUPLING  VARIABLE  ON  THE  SYSTEM  J  SIDE  OF  THE  NETWORK, 
AND  e.  REPRESENTS  THE  INSTANTANEOUS  ERROR  IN  THE  /-TH 
COUPLING  VARIABLE. 

associated  with  integration  over  a  network  (e.g.,  network 
delays,  communication  bandwidth,  filters,  sampling,  etc.),  the 
instantaneous  value  of  the  coupling  variable  i  will  not  be  the 
same  on  the  two  sides  of  the  network.  Let  c,  j  {t)  represent  the 
value  of  c,  as  seen  by  System  j  at  time  t .  Ideally,  i.e.,  if  the 
systems  were  co-located  and  coupled  physically,  we  would 
have  C; ,  (t)  =  c,  j  (i) .  In  the  ID-HIL  case,  however,  the  two 
instances  of  the  variable  are  not  the  same,  i.e.,  c.y{t)  . 

Thus,  we  propose  to  use  the  difference  between  q. ,  and  c.  ^  as 
a  metric  for  fidelity  in  the  coupling  variable  i ;  i.e., 

^2  (0  ~  ^2,1  (0  ~  ^2,2  (O' 

Note  that  causality  is  taken  into  account  while  defining  the 
error  signals  in  (1)  when  using  the  system  output  instances  of 
the  variables  (i.e.,  c,  2  and  Cj  1 )  as  the  references  for  the  system 
input  instances  of  the  same  variables  (i.e.,  c,  j  and  Cjj). 

It  must  also  be  emphasized  that  increasing  fidelity  of  one 
of  the  coupling  signals  does  not  automatically  imply  an 
increase  in  the  fidelity  of  the  remaining  coupling  signals,  as 
shown  in  [32,  33].  Thus,  in  general,  no  single  coupling  signal 
error  can  completely  describe  the  system  fidelity  by  itself 
Therefore,  to  increase  the  system  fidelity,  this  error  needs  to  be 
driven  to  zero  for  all  coupling  variables.  With  respect  to  Fig.  1, 
for  example,  this  implies  e^  :=  Cj  2  -  c, ,  — >  0  and 

.—  £2,1  ~  ^2,2  ^  ®  • 

To  characterize  system  fidelity,  all  coupling  signal  errors 
can  be  aggregated  into  a  single  error  metric.  One  way  to 
achieve  this  is  to  use  the  following  norm  of  weighted  error 
norms: 


where  n  is  the  total  number  of  coupling  signals.  The  goal  of 
maximizing  the  system  fidelity  then  translates  to  minimizing 
the  system-level  error  metric  E  . 

The  rationale  behind  using  different  weights  for  different 
error  vectors  is  that  the  error  in  one  coupling  signal  may  be 
more  critical  than  the  error  in  another  for  a  particular  output  of 
interest.  In  this  paper,  we  will  consider  the  weight 


i.e.,  the  norm  of  error  of  each  coupling  variable  will  be 
normalized  with  respect  to  its  own  initial  value. 

III.  IMPROVING  FIDELITY  USING  ITERATIVE 
LEARNING  CONTROL 

The  fundamental  problem  with  the  proposed  fidelity  metric 
is  that  it  is  not  available  online;  i.e.,  the  value  of  c.  ^  and  c.  ^  at 
a  given  instant  cannot  be  made  available  for  the  error 
calculation  at  the  same  instant.  The  error  e^(t)  can  be  known 
only  at  t  +  z,  i.e.,  after  some  delay  z  that  is  needed  to 
collocate  the  measurements  c, ,  (t)  and  c,  2  (f) .  Thus,  the  error 
cannot  be  used  without  delay  to  make  online  corrections  to 
improve  fidelity  while  an  experiment  is  running.  Even  though 
the  signal  could  be  made  available  with  a  delay,  so  that  a 
feedback  loop  could  be  closed  around  it,  since  the  overarching 
goal  of  this  effort  is  to  go  beyond  the  fundamental  limitations 
of  such  feedback  methods,  this  option  will  not  be  discussed  any 
further  in  this  paper. 

Instead,  this  work  considers  an  offline  application  of  the 
described  metric  to  improve  fidelity.  To  achieve  this,  the 
iterative  learning  control  (ILC)  paradigm  is  leveraged  to 
improve  fidelity  of  an  ID-HIL  experiment  iteratively.  The  rest 
of  this  section  describes  this  framework. 

The  ILC -based  framework  used  in  this  work  is  illustrated 
in  Fig.  2  for  one  of  the  coupling  signals  as  an  example.  The 
error  as  defined  in  Eq.  (1)  is  provided  offline  as  the  error  signal 
to  the  ILC  algorithm.  Together  with  the  control  input  used  in 
the  corresponding  run,  the  algorithm  then  shapes  the  control 
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FIG.  2.  PROPOSED  ILC-BASED  FRAMEWORK  FOR  ITERATIVELY 
IMPROVING  ID-HIL  FIDELITY.  THIS  FIGURE  ILLUSTRATES  A 
DECENTRALIZED  APPROACH  IN  WHICH  AN  INDEPENDENT  ILC 
CONTROLLER  IS  UTILIZED  FOR  EACH  COUPLING  SIGNAL. 


input  according  to  a  learning  algorithm  to  attenuate  the  error  in 
the  next  run,  i.e., 

ur'=f(ur,er),  (4) 


with 


ur=[Mr(o)  <-(1)  - 

er=[e/(0)  <(1)  -  er(iV)7, 


(5) 


where  the  superscript  m  is  an  index  for  the  iteration,  and  N 
represents  the  number  of  time  steps. 

It  is  very  important  to  emphasize  here  that  this  problem  is 
not  as  trivial  as  using  the  error  from  the  initial  mn  and  adding  it 
to  the  coupling  variable  in  the  current  run.  The  reason  is  the 
bidirectional  nature  of  the  coupling  between  the  systems,  which 
prevents  the  applicability  of  this  trivial  solution.  For  example, 
referring  to  the  example  in  Fig.  2,  any  modification  to  c"'*'(k) 
will  propagate  through  System  1,  Network,  and  System  2  and 
affect  c"7‘ («),  n  =  k  +  \,k  +  2,...,N  .  In  other  words,  if  c,  2  is 
regarded  as  a  reference  for  Cj ,  in  the  ILC  framework,  then  the 
problem  can  be  considered  as  the  reference  signal  changing  as  a 
result  of  the  ILC  action. 

Thus,  designing  the  learning  frinction  f(u'",e-")  properly 
is  crucial.  Different  learning  functions  could  be  considered  for 
their  suitability  to  ID-HIL.  In  this  paper,  the  ILC  learning 
algorithm  of  the  form 

iC  [k)  =  e(v)(«r  {k)  +  L{q)(^'  (A  +  1))  (6) 


is  considered,  since  it  was  reported  to  be  a  widely  used 
algorithm  [44].  In  (6),  q  represents  the  forward  time  shift 
operator.  Due  to  their  applicability  to  nonlinear  systems  and 
wide  use  [44],  PD-type  learning  functions  are  considered  here, 
e.g., 

ur  {k) = u:  {k) + k^er  {k)+k,  (<  (a)  (a  -i))  (?) 

This  type  of  learning  functions  also  has  the  advantage  of  not 
requiring  a  model  of  the  system  as  part  of  the  design  process 
[45-54],  which  is  particularly  suitable  for  the  ID-HIL  paradigm 
and  the  specific  goals  of  this  paper. 

This  ILC -based  framework  is  applied  to  each  coupling 
signal  independently.  In  this  framework,  each  coupling  signal 
has  its  own  ILC  controller  (with  its  own  learning  frinction)  that 
does  not  communicate  with  the  ILC  controllers  of  the 
remaining  coupling  signals.  Through  such  a  decentralized 
approach,  scalability  could  be  achieved;  i.e.,  the  method  would 
be  easy  to  extend  to  ID-HIL  setups  with  multiple  sites. 


IV.  CASE  STUDY 

To  investigate  the  viability  of  the  framework  described  in 
Sections  II  and  III,  a  completely  simulation-based  study  is 
performed  with  a  vehicle  and  driver  system.  A  high-level 
overview  of  the  system  is  shown  in  Fig.  3.  The  major 
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FIG.  3.  THE  OUTLINE  OF  THE  SYSTEM  CONSIDERED  IN  THE 
PRELIMINARY  STUDY 


components  are  the  Driver,  Vehicle,  Internet,  Drivetrain, 
Flywheel,  Idle  Controller,  and  Engine.  The  Driver  and  Vehicle 
subsystems  constitute  System  1,  whereas  System  2  comprises 
the  Drivetrain,  Flywheel,  Engine,  and  Idle  Controller.  The  two 
Systems  are  connected  over  the  Internet  through  the  coupling 
signals  throttle,  transmission  shaft  torque  and  transmission 
shaft  speed.  Hence,  three  instances  of  the  ILC  framework 
shown  in  Fig.  2  are  employed  for  the  three  coupling  signals. 

This  section  first  gives  the  details  of  these  components  and 
then  presents  the  results  for  the  ILC -based  fidelity 
improvement. 


System  Description 

The  driver  model  is  a  PI  controller  with  saturation  and  anti¬ 
windup.  It  takes  the  difference  between  the  desired  and  actual 
vehicle  velocities  as  inputs  and  generates  an  output  within  the 
interval  [-1,1],  where  positive  and  negative  values  correspond 
to  throttle  and  brake  commands,  respectively. 

The  vehicle  dynamics  model  is  a  point  mass  representation 
of  a  military  vehicle  (the  High  Mobility  Multipurpose  Wheeled 
Vehicle  -  HMMWV)  and  includes  differentials,  wheel  inertia,  a 
Coulomb  and  viscous  friction  based  brake  model,  rolling 
resistance,  aerodynamic  drag,  and  tire  slip.  The  model  can  be 
expressed  in  the  following  form 


f 

,  wheel 

\ 

/ 

\ 

^'’■vehicle 

x  = 

h  ^ 

differential  ^ 

'^brake 

V 

'wheel 

) 

\  ^wheel 

) 

(8) 


■f rolling  f ae 


where  is  vehicle  mass,  J is  wheel  inertia,  b  is 

wheel  viscous  damping  coefficient,  is  wheel  radius,  and 
X  is  vehicle  acceleration.  Furthermore,  and  are 

aerodynamic  and  rolling  resistance  forces,  respectively,  and 
differential  are  differential  and  brake  torques, 

respectively,  and  they  are  given  by 

faero=^^PairCd\Ax  (9) 

where  A  is  the  vehicle  frontal  area,  is  the  air  density, 
is  the  aerodynamic  drag  coefficient,  and  x  is  vehicle  velocity; 
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FIG.  4.  THE  ENGINE  MODEL. 


FIG.  5.  THE  STATIC  ENGINE  MAP. 


f rolling  =Sgn[x) 
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where  a,  are  empirical  coefficients,  and  P  is  the  tire  pressure; 
and 
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where  is  brake  viscous  damping  coefficient,  is  the 

static  Coulomb  force,  and  is  the  brake  command  from  the 
driver. 

A  map-based  engine  model  is  considered  as  shown  in  Fig. 
4.  The  map  is  a  static  map  obtained  experimentally  from  a 
physical  6L  V8  diesel  engine.  The  input  to  the  map  is  the  fuel 
rate  and  the  engine  speed,  and  the  output  is  the  engine  torque. 
Fig.  5  shows  the  engine  map. 

The  fuel  rate  is  given  by  the  fuel  controller,  which  fulfills 
two  purposes.  First,  it  implements  the  turbo  lag  as  a  first  order 
system  of  the  form  k !  {rs  +  l) .  The  input  to  the  turbo  lag  is  the 
difference  between  the  maximum  fuel  rate  for  the  given  engine 
RPM  and  the  naturally  aspirated  fuel  rate,  both  determined 
experimentally.  The  output  of  the  turbo  lag  is  added  to  the 
naturally  aspirated  fuel  rate.  This  sum  is  then  compared  to  the 
maximum  possible  fuel  rate  for  the  given  engine  RPM  and 
throttle,  and  the  minimum  of  the  two  is  taken  as  the  unadjusted 
fuel  rate.  Second,  the  fuel  controller  monitors  the  unadjusted 
fuel  rate  and  adjusts  it  if  the  engine  speed  falls  below  650  RPM 
or  exceeds  the  maximum  rated  speed  of  3300  RPM  to  bring  the 
speed  back  to  the  desired  operating  region. 

The  idle  controller  is  a  PI  controller  with  saturation  and 
anti-windup,  and  is  activated  when  the  throttle  demand  from 


the  driver  falls  below  11%  to  maintain  an  engine  idle  speed  of 
750  RPM. 


The  flywheel  model  is  an  inertia  element  that  takes  the 
load  torque  and  the  engine  torque  as  inputs  and  determines  the 
pump/engine  speed  as  the  output.  This  speed  signal  is  fed  to  the 
drivetrain  model  as  the  pump  speed,  as  well  as  to  the  engine 
and  idle  controller  as  the  engine  speed. 

The  drivetrain  model  includes  the  torque  converter, 
transmission,  and  shift  logic.  The  torque  converter  model  is  a 
static  model  that  takes  pump  and  turbine  speeds  as  inputs  and 
generates  pump  and  turbine  torques  according  to  the  equations 


-(®.) 


sign[l-co^) 


(12) 


where  is  the  speed  ratio  between  turbine 

and  pump  speeds,  is  a  piecewise  function 

approximating  a  desired  capacity  factor  curve,  and  «(<»,.)  is  a 
piecewise  linear  function  approximating  a  desired  torque  ratio 
curve.  The  pump  side  of  the  torque  converter  is  connected  to 
the  engine  flywheel;  thus,  pump  speed  and  engine  speed  are  the 
same.  The  turbine  side  of  the  torque  converter,  on  the  other 
hand,  is  connected  to  the  transmission  model. 

The  transmission  model  takes  into  account  the 
transmission  shaft  inertia,  stiffness,  and  damping,  as  well  as  the 
gear  inefficiencies  and  torque  losses  due  to  fluid  churning. 
Specifically,  the  speed  reduction  in  each  gear  is  assumed  to  be 
ideal,  while  the  torque  multiplication  is  assumed  to  be  scaled  by 
an  efficiency  factor.  Furthermore,  the  torque  lost  due  to  fluid 
churning  is  modeled  as  variable  nonlinear  resistance  of  the 
form 


'^chi'uning  loss  q  +r^  {gear)  (13) 

where  q  and  are  coefficients  that  change  depending  on  the 
gear. 

The  inputs  to  the  shift  logic,  the  final  element  in  the 
drivetrain  model,  are  the  transmission  output  shaft  speed  and 
the  throttle  demanded  by  the  driver.  The  simple  chart  shown  in 


Fig.  6  is  used  to  determine  if  a  shift  is  to  be  initiated.  The  solid 
and  dashed  lines  in  Fig.  6  indicate  upshift  and  downshift 
thresholds,  respectively.  Note  that  this  chart  is  only  a  crude 
approximation  of  a  real  shift  map,  but  it  is  employed  here  for 
simplicity. 

Modeling  the  Networking  of  the  Systems 

This  section  describes  the  architecture  used  for  simulating 
the  communication  of  the  coupling  variables  over  the  Internet 
between  System  1  and  2.  In  the  adopted  framework,  the  System 
1  acts  as  the  client  and  the  System  2  acts  as  the  server.  System  1 
sends  updated  transmission  speed  and  throttle  signals  at  a 
frequency  of  20  Hz,  regardless  of  whether  it  receives  a  response 
or  not.  System  2,  on  the  other  hand,  only  responds  to  the 
packets  it  receives,  i.e.,  it  only  sends  an  updated  transmission 
torque  signal  when  it  receives  a  packet  from  System  1.  Fig.  7 
and  8  illustrate  the  communication  flowcharts  for  System  1  and 
2,  respectively. 

The  Internet  is  modeled  to  allow  for  an  optimization  based 
tuning  of  the  ILC  parameters,  as  well  as  to  avoid  clock 
synchronization  issues  between  the  two  Systems. 

To  develop  a  representative  model  of  the  Internet,  the 
network  between  two  computers  in  Ann  Arbor,  MI,  and  Warren, 
MI,  was  characterized  through  a  series  of  experiments,  in  which 
packets  ranging  from  64  bytes  to  1024  bytes  were  exchanged 


FIG.  7.  COMMUNICATION  FLOWCHART  FOR  THE  SYSTEM  1 


1.2 


FIG.  9.  CHARACTERIZATION  OF  NETWORK  QUALITY  OF  SERVICE 
BETWEEN  TWO  COMPUTERS  IN  ANN  ARBOR,  Ml,  AND  WARREN,  Ml. 


TABLE  1.  STATISTICS  FOR  THE  RESULTS  IN  FIG.  9 


Number  of  packets 

1,110,000 

Packet  size 

1024  B 

Avg.  delay 

25.7  ms 

Min  delay 

25.3  ms 

No  of  spikes 

114(0.01%) 

No  of  drops 

56  (0.005%) 

on  different  days  and  at  different  times  of  day  using  the  UDP/IP 
protocol.  This  protocol  is  preferred  for  its  speed,  but  does  not 
guarantee  packet  delivery.  A  typical  result  for  round  trip  time 
delay  vs.  time  of  day  obtained  from  one  those  experiments  is 
shown  in  Fig.  9.  The  figure  clearly  shows  a  multi-modal 
character  in  the  sense  that  some  packets  experience  a  delay 
around  25  ms,  and  some  around  350ms  (spikes),  while  others 
are  dropped  (shown  as  zero  delay  in  the  figure).  A  packet  is 
considered  dropped  in  this  case  if  it  does  not  arrive  within  Is. 
Table  1  provides  some  statistics  of  the  results  shown  in  Fig.  9. 

Based  on  the  statistics  given  in  Table  1,  the  percentages  of 
spikes  and  dropped  packets  are  very  small.  Thus,  as  a  first 
approximation,  the  spikes  and  drops  are  neglected,  and  the 
emphasis  is  given  to  the  characterization  of  the  dominant  mode 
of  the  delay,  which  appears  in  the  darker  potion  of  the  curve  in 
Fig.  9  around  25ms  delay. 

For  the  purposes  of  this  study,  the  delay  is  assumed  to  be 
an  independent  random  variable.  The  assumption  of 
independence  is  validated  by  checking  the  autocorrelation  of 
the  observed  delay  sequence  after  the  average  delay  is 
subtracted  (Fig.  10).  The  fact  that  the  autocorrelation  is  almost 
zero  for  all  lags  except  for  zero  indicates  that  there  is  no 
correlation  between  the  delay  values,  thereby  justifying  the 
assumption  of  independence. 

The  histogram  of  the  dominant  mode  of  delay  is  shown  in 
Fig.  11.  The  figure  also  shows  that  the  dominant  mode  can  be 
approximated  quite  accurately  with  a  lognormal  distribution 
with  parameters  =  -0.9  and  ct„  =  1 ,  where  the  subscript  rt 
indicates  round  trip.  A  scaling  factor  of  0.03/70  is  used,  along 
with  an  offset  of  0.0254  s.  Thus,  the  modeled  distribution  in 
Fig.  1 1  comes  from  a  random  variable  of  the  form 
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FIG.  10.  AUTOCORRELATION  OF  DELAY. 
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FIG.  1 1 .  ACTUAL  VS.  MODELED  DISTRIBUTION  OF  DELAY  BETWEEN 
ANN  ARBOR  AND  WARREN. 

-0.0254 +  ^Log-N(//,,  =-0.9, a,,  =l)  (14) 

The  model  given  in  Eq.  (14)  is  for  the  round  trip  time 
delay.  However,  in  the  simulation,  a  model  for  the  one-way 
delay  is  needed.  To  find  the  parameters  for  the  one-way  delay 
model,  first  recall  that  for  two  independent  random  variables 
the  mean  of  their  sum  is  the  sum  of  their  means,  and  the 
variance  of  their  sum  or  difference  is  the  sum  of  their  variances. 
Second,  recall  that  the  mean  of  a  lognormal  distribution  is 
given  by  ,  and  the  variance  is  -1) .  Finally, 

assume  that  the  network  characteristics  are  the  same  both  ways. 
Then,  the  parameters  and  for  the  one-way  delay 
model  can  be  found  by  solving 

\  (15) 

This  results  in  the  following  one-way  delay  model 
^  0.0254 

^  ow  ^  « 

(16) 

0  03 

+  —Log-N(//„,..  =-1.838,rT„„  =1.221) 

and  the  sum  of  two  such  one-way  delays  results  in  the  desired 
distribution  for  the  round  trip  delay. 
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TABLE  3.  ILC  PARAMETERS  USED  IN  THE  CASE  STUDY 
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Coupling 

variable 

ILC  parameters  for 
Ann  Arbor  -  Warren 

ILC  parameters  for 
Michigan  -  California 

Torque 

=0.697;  =0.496 

A^  =0.197;  k,  =0.084 

Speed 

Ap  =0.735;  A^  =0.138 

A^  =  0.274;  k,  =  0.005 

Throttle 

Ap  =0.719;  A^  =0.004 

Ap  =0.503;  A^  =0.031 

that  sought  to  minimize  the  system  error  as  given  in  Eq.  (2) 
after  10  ILC  iterations.  The  resulting  parameters  are  shown  in 
Table  3. 


Delay  (s) 

FIG.  12.  ACTUAL  VS.  MODELED  DISTRIBUTION  OF  DELAY  BETWEEN 
MICHIGAN  AND  CALIFORNIA. 

To  obtain  a  model  that  represents  a  longer  connection,  the 
network  characterization  given  in  detail  above  for  an  Ann  Arbor 
-  Warren  connection  was  repeated  for  a  connection  between 
Michigan  and  California.  The  actual  and  model-based 
distributions  are  given  in  Fig.  12.  The  one-way  delay  model  in 
this  case  becomes 

0.1068 

ow  « 

1  083 

+  ^Log-N(//„,  =-1.838, =1.221) 

Simulation  Results 

The  ILC  framework  shown  in  Fig.  2  was  used 
independently  for  all  the  coupling  variables,  i.e.,  the  throttle, 
shaft  torque,  and  shaft  speed.  All  three  ILC  controllers 
independently  used  the  learning  algorithm 

[k)  =  <  (A)  -r  (k)  +  k,  (e,™  (k)  -  ef  (A  -l))  (18) 

with  and  k^  being  control  parameters  that  can  be  tuned 
independently  for  each  controller.  Thus,  for  this  case  study,  the 
ILC  framework  variables  can  be  defined  as  in  Table  2.  The  ILC 
action  for  shaft  torque  is  injected  on  the  System  1  side  of  the 
Internet,  whereas  the  ILC  actions  for  shaft  speed  and  throttle 
are  injected  on  the  System  2  side. 

First  set  of  simulations  were  performed  for  the  first  130 
seconds  of  the  FTP75  drive  cycle.  The  control  parameters  were 
tuned  for  each  delay  model  separately  using  a  genetic  optimizer 


TABLE  2.  VARIABLES  IN  THE  ILC  FRAMEWORK  FOR  THE  CASE  STUDY 


Variable 

Description 

^1../ 

Shaft  torque  on  System  j  side,  7  =  1,2 

ei 

Error  in  shaft  torque,  gj  =  Cj  2  -  Cj  j 

^2.J 

Shaft  speed  on  System  j  side,  7  =  1,2 

ei 

Error  in  shaft  torque,  gj  =  ^2,1  -  C2,2 

Throttle  on  System  j  side,  7  =  L2 

^3 

Error  in  throttle,  gs  =  C3 1  -  C3  2 

A  representative  result  for  the  Ann  Arbor  -  Warren  delay 
scenario  is  summarized  in  Fig.  13,  which  shows  the  ILC 
performance  of  the  three  coupling  variables  for  10  ILC 
iterations.  The  figure  is  representative  in  the  sense  that  each 
time  it  is  re-generated,  the  curves  would  look  slightly  different 
due  to  the  random  nature  of  delay.  Nevertheless,  the 
performance  level  stays  the  same.  Specifically,  the  figure  shows 
a  dramatic  reduction  in  the  errors  in  coupling  variables.  More 
than  95%  reduction  is  achieved  in  the  normalized  2-norms  of 
errors  of  all  coupling  variables,  where  errors  are  normalized 
with  respect  to  the  2-norms  of  the  corresponding  initial  errors 
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FIG.  13.  REPRESENTATIVE  ILC  PERFORMANCE  FOR  THE  THREE 
COUPLING  VARIABLES  FOR  THE  ANN  ARBOR  -  WARREN  DELAY. 
LOWER  ERROR  VALUES  MEAN  HIGHER  FIDELITY. 
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FIG.  14.  REPRESENTATIVE  ILC  PERFORMANCE  FOR  THE  THREE 
COUPLING  VARIABLES  FOR  THE  MICHIGAN  -  CALIFORNIA  DELAY. 
LOWER  ERROR  VALUES  MEAN  HIGHER  FIDELITY. 
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FIG.  15.  REPRESENTATIVE  ILC  PERFORMANCE  FOR  THE  130-340  SEC 
WINDOW  OF  THE  FTP75  AND  FOR  THE  ANN  ARBOR  -  WARREN  DELAY. 


FIG.  16.  REPRESENTATIVE  ILC  PERFORMANCE  FOR  THE  130-340  SEC 
WINDOW  OF  THE  FTP75  AND  FOR  THE  MICHIGAN  -  CALIFORNIA 
DELAY. 


as  given  in  Eq.  (3). 

Figure  14  shows  a  representative  ILC  performance  for  the 
Ml  -  CA  delay  model.  As  a  result  of  the  larger  variation  in 
delay,  the  performance  in  terms  of  reducing  the  error  within  10 
iterations  is  not  as  good  as  in  the  previous  case.  Nevertheless, 
the  ILC  is  still  capable  of  reducing  the  error  by  around  90%  for 
speed  and  throttle,  and  around  60%  for  torque.  This  result  also 
highlights  the  fact  that  different  coupling  variables  may  present 
different  levels  of  challenge  for  improving  their  fidelity,  a  result 
consistent  with  previously  published  results  [22,  23,  32,  33].  In 
this  particular  scenario,  the  torque  signal  is  more  sensitive  to 
delay  than  the  speed  and  throttle  signals,  since  the  speed  signal 
is  filtered  through  vehicle  inertia  and  the  throttle  signal  is 
closely  related  to  speed.  Hence,  it  is  harder  to  improve  the 
fidelity  of  the  torque  signal. 

A  comparison  of  the  ILC  parameters  for  the  two  delay 
conditions  shows  that  the  optimization  reduces  the  proportional 
gains  for  all  variables  in  the  MI  -  CA  delay  scenario  to  cope 
with  the  increased  variation  in  delay  through  a  slower  learning. 
However,  it  is  interesting  that  the  reduction  in  proportional  gain 
is  not  to  the  same  extent  for  the  throttle  signal  as  for  the  torque 


and  speed  signals.  This  again  emphasizes  the  different 
characteristics  different  coupling  variables  may  exhibit. 

Figures  15  and  16  show  representative  ILC  performances 
for  a  different  drive  cycle  than  what  was  used  for  ILC 
parameter  tuning.  Specifically,  the  130s-340s  time  window  of 
the  FTP75  was  used  in  this  part  of  the  study,  which  exhibits  a 
more  aggressive  driving  behavior  than  the  0s-130s  time 
window.  The  figures  show  that  the  ILC  performance  levels 
observed  for  the  tuning  drive  cycle  are  still  maintained.  Thus, 
the  proposed  ILC  framework  seems  to  be  robust  to  the  drive 
cycle. 

These  results  highlight  the  merit  and  potential  of  the 
proposed  framework  to  improve  fidelity,  and  encourage  the 
further  development  of  this  framework.  Some  specific 
questions  of  interest  are  what  the  system  requirements  are  for 
the  proposed  framework  to  work,  what  type  of  learning 
functions  are  best  suited  for  this  application,  how  the  ILC 
parameters  relate  to  system  characteristics,  and  what  the 
robustness  of  this  approach  is  and  how  it  can  be  maximized.  In 
terms  of  robustness,  the  robustness  to  variations  in  the  initial 
conditions  when  actual  hardware  is  introduced  and  the 
robustness  to  variations  in  the  network  delays  are  of  particular 
importance.  Potential  future  work  also  includes  a  centralized 
ILC  approach,  as  opposed  to  the  decentralized  approached  used 
in  this  study,  where  every  coupling  variable  has  its  independent 
controller.  A  centralized  approach  could  yield  a  different 
performance  in  terms  of  final  error,  convergence  speed,  or 
robustness,  which  may,  however,  come  at  the  expense  of  ease 
of  scalability. 

V.  SUMMARY  AND  CONCLUSIONS 

An  Iterative  Learning  Control  based  framework  has  been 
developed  to  improve  fidelity  in  Internet-distributed  hardware- 
in-the-loop  simulation.  The  framework  has  been  tested  using  a 
purely  simulation-based  case  study  that  showed  that  60-95% 
reduction  in  coupling  variable  errors  can  be  obtained  with  the 
proposed  method.  The  results  imply  that  the  framework  can 
improve  the  fidelity  of  the  networked  simulation  significantly, 
and  encourage  further  development  and  experimentation  with 
actual  hardware  in  the  loop. 
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